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Abstract 

Evaluation of the quality of Arabic education in Malaysia needs to be conducted on a continuous basis to achieve 
the objectives of Arabic education. Student Evaluation of Education Quality (SEEQ) was proposed as one of the 
evaluation instruments of the quality of Arabic education. This study aimed to evaluate the usability of SEEQ in 
the context of Malaysian education, including its validity and reliability based on the collected local data. In this 
study, a total of 491 samples from Islamic National Secondary Schools (SMK(A)) throughout Malaysia were 
involved. Confirmatory Factor Analysis (CFA) and Cronbach’s alpha for internal consistency were performed 
using AMOS and SPSS.A new model of SEEQ (8 constructs and 25 of 31 items were retained after 
modifications) with good readings off it indices and alpha value was obtained. This study suggests using the new 
model of SEEQ as the indicator of the quality of Arabic education in Malaysia. 

Keywords: SEEQ, Confirmatory Factor Analysis (CFA), Student Evaluations of Teaching (SET), Arabic and 
internal consistency 

1. Introduction 

The history of Arabic education in Malaysia began with the arrival of Islam in this region. The residents of Tanah 
Melayu had first recognised the Arabic letters and its enunciation through Islamic religious practices that was 
related to Arabic. However, during that time, Arabic was not intensely taught and studied since the focus was 
given further to the teaching of Al-Quran and Fardhu Ain (Khoo, 1980). As such, Arabic was taught per se 
through religious studies that using books written in Arabic (Rejab, 1992). 

It can be said that the formal Arabic studies in Malaysia began with the emergence of hut schools in the 18th 
century (Jusuh, 1990). Initially, the study focused on the study of syntax and morphology that served as a 
pre-requirement for the study of Islamic knowledge learned afterward (Samah, 2009).In 1996, through the 
Education Act 1996, Arabic was officially included in the Malaysian educational curriculum as a foreign 
language subject and was taught in schools as one of the linguistic subjects. Today, Arabic is not only taught in 
Arabic stream schools anymore since it has also been made available as an elective subject in boarding schools 
and higher education institutions. The learning and teaching of Arabic were further expanded with the 
establishment of the departments of Arabic language in public higher education institutions. Universiti 
Kebangsaan Malaysia was the first higher learning institution to establish the Department of Arabic Studies and 
Islamic Civilisation, followed by International Islamic University Malaysia through the Department of Arabic 
Language, and then University of Malaya through the Department of Arabic Language of the Faculty of 
Language and Linguistics. Universiti Putra Malaysia has also established its Arabic Language Unit and 
subsequently in 2005, Universiti Sains Islam Malaysia established its Arabic Language and Communication 
Programme in its Faculty of Major Language Studies. The last one was the same establishment in Universiti 
Sultan Zainal Abidin (UniSZA). 

These favourable situations had placed Arabic in a special situation. Asma Omar (in Mezah, 2009) asserted that 
it is difficult to treat Arabic as a foreign language because of its status in Islamic society in Malaysia. Therefore, 
Arabic has its own status compared to other foreign languages based on the observational history of Islamic 
studies, culture and national education policy. 
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2. Problem Statement 

Despite the robust growth of teaching and learning Arabic in all levels from primary education to university level 
throughout Malaysia, there are a few shortcomings which have become an open secret among the researchers of 
Arabic education and has also been acknowledged by the authorities. This weakness often to be a topic of 
discussion in academic research and educational seminars (Mezah, 2009; Rahimi, Ghani, & Abdullah, 2008). For 
example, studies done by Abdullah (2009), Mezah (2009) and Jacob (2007) on the quality of Arabic essay 
writing in the secondary and university levels had found that the levels of student performance were less than 
satisfactory and need attention. The weaknesses were particularly in terms of vocabulary such as inadequate 
vocabulary, inappropriate choice of words in sentence context, misspellings, word order not following the right 
methods, redundant use of words, convoluted sentences and a lot more. This study, among others things, 
revealed that the rooted cause is the abandonment of morphology aspects from both teachers and students when 
teaching and learning Arabic. Moreover, the attitude, enthusiasm and initiative factors of the Arabic students 
themselves had also contributed to this drawback. 

Language is a skill. In this case, the Arabic’s fundamental aspects that should be mastered by the students have 
also suffered a similar problem from its mastery aspect. A study done by Rahimi et al. (2006) among the 
religious secondary school students of Yayasan Islam Kelantan had described this situation. A total of 1665 form 
two students from 16 religious secondary schools throughout Kelantan had been selected as respondents. The 
study was conducted on five aspects of listening skill; (i) sub-skill of listening comprehension of the Arabic 
enunciation; (ii) sub-skill of listening comprehension of the lexical meaning of words; (iii) sub-skill of listening 
comprehension of the contextual meaning of words; (iv) sub-skill of listening comprehension of literal texts; and 
(v) sub-skill of listening comprehension of inferential texts. Findings of the study showed that the first sub-skill 
was at a high level; the second and third sub-skills were average; and the fourth and fifth sub-skills were at poor 
levels. These data showed that the listening level was at a danger level. Rahimi et al. (2006) also stressed that 
these weaknesses had indirectly affected the other skills based on the study done on speaking skills by Nik 
Rahimi in year 2000.This condition is following the basis that listening skill is regarded as the first level of the 
four language skills in learning taxonomy. 

Based on the external and internal weaknesses of the Arabic students, these studies and others had provided early 
indicators of the needs for revaluation of the Arabic education ecosystem especially in secondary schools (see, 
Mohamad, 2009; Teh et al., 2010; Rahman & Ahman, 2007; Jabar, 2004; Ismail & Samah, 2007; Samah & Bakar, 
2008; Samah & Ismail, 2006). Compared to other levels of Arabic education, the secondary level obtained 
numerous research focus that revealed these weaknesses. The evaluation and quality assurance process for the 
performance of the education quality is one of the important efforts implemented in each education system 
throughout the world. Malaysia, under the supervision of the Ministry of Education has started the process of 
evaluation and quality control of the Arabic education with various public initiatives such as the establishment of 
congregation unit of inspectors and quality assurance, the standard formation of Malaysian teachers, the 
appointment of excellent teachers based on subjects, regular teaching supervisions and others. 

Evaluation instrument is one of the efforts to improve and control the quality of education. Instruments become 
the popular choice among educational researches especially with the students’ participation as evaluators. This 
field is known as Student Evaluation of Teaching (SET). Student Evaluation of Education Quality (SEEQ) that 
was developed by Marsh (1982) is one of the most popular instruments of SET in educational world which has 
been implemented in a few countries. SEEQ is developed with the consideration of multiple dimensions that are 
present in teaching and learning, thus making it suitable to be used for all educational subjects. Therefore, it is 
widely utilised all over the world and considered to be one of the most reliable and suitable instruments to 
measure the effectiveness of teaching. The experts’ recognition of the SEEQ questionnaire and the 
comprehensiveness of its measuring parameters encourage researchers to also implement it in Malaysia. The use 
of SEEQ in the context of Arabic education is expected to help in the process of evaluation and quality control 
with regards to other evaluation method by the other parties. However, the development of an instrument will not 
stop in the development phase, but continues throughout all phases, places and time. The existing instrument 
such as SEEQ needs to go through the psychometrics measurement evaluation process such as reliability, 
validity and quality of the items for it to remain relevant. These tests are more likely in the event of relocation, 
changes in respondents and time of study compared to the original condition of the instrument (Cohen, Manion, 
& Morrison, 2013). 
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3. Literature Review 

3.1 Student Evaluation of Education Quality (SEEQ) 

Student Evaluation of Education Quality (SEEQ) is one of the standard instruments that takes into account the 
various dimensions in the Student Evalution of Teaching (SET) field. Marsh (1982) developed SEEQ in 
multi-dimensional form and used it for the first time in the United States of America. Since then, SEEQ has been 
widely used around the world and is considered as one of the most reliable and suitable instruments for 
measuring the effectiveness of teaching (Coffey & Gibbs, 2001).For instance, Marsh (1982) had reported the 
internal consistency of SEEQ scale in the United States as between 0.88 to 0.97. In the Australian context, the 
internal consistency of SEEQ was 0.89 (Marsh & Roche, 1993). Meanwhile, Al-Hinai (2011) in his study in 
Oman had found the Cronbach’s alpha value of 0.87. Moreover, SEEQ had been cross-cultural and its usage was 
practised and proposed in various countries such as England (Coffey & Gibbs, 2001), Spain (Marsh, Touron, & 
Wheeler, 1985), Thailand (Jarernvongrayab, 2007), China (Marsh, Hau, Chung, & Siu, 1997, 1998), Papua New 
Guinea, India, Nigeria, Flipina, Flong Kong and many more (Perry & Smart, 2007). According to Perry and 
Smart (2007), the study that used SEEQ across the border of the United States had supported the usability factor 
contained in SEEQ outside its original context. Apart from being able to be implemented across nations, SEEQ 
has also been proven to be able to be implemented across various educational subjects including foreign 
languages. The suitability of the application of SEEQ in the teaching of a foreign language was proven in the 
Al-Elinai (2011) study. This cross-subject implementation capability is one of the selection criteria rendering its 
use in the assessment of the teaching of Arabic as a foreign language in Malaysia. 

Thus, the experts’ recognition of the SEEQ questionnaire as well as its suitability to the educational assessment 
has led the researchers to use it as the measurement and quality control tool for the Arabic education in Malaysia. 
The use of the existing questionnaire is the chosen method as it is the result of expert’s work compared to the 
self-developed questionnaire that could lead to various problems (Idris, 2010). This study is the first to use 
SEEQ in the context of education in Malaysia. 

As other questionnaires in the Student Evaluation of Teaching (SET), SEEQ uses various dimensions and aspects 
to evaluate teaching. There are 9 dimensions that are taken into account, including the overall evaluation of the 
teacher and the subject. Multidimensionality attributed to the educational field or the social sciences that in 
general dealing with complex constructs. The dimensions in SEEQ are; learning value, teacher enthusiasm, 
organisation, group interaction, individual rapport, breadth of coverage, assessment, assignments and overall 
rating. All dimensions are measured by a five-point Likert scale which are; 1-very poor, 2-poor, 3-moderate, 
4-good, 5-very good. 

Dimension 1: Learning Value (4 items): This scale consists of four items that measure whether the students 
found the classes stimulating their mind and whether their interest in the subject increased or not. 

Dimension 2: Teacher Enthusiasm (4 items): This scale asks the students to evaluate their teachers’ morale and 
whether he or she is able to attract students’ attention with his or her method of teaching. 

Dimension 3: Organisation (4 items): This scale asks whether the course materials are prepared properly and 
presented clearly, and whether the objectives of the course are met accordingly. 

Dimension 4: Group Interaction (4 items): This scale asks the students to evaluate their teachers’ ability to 
encourage students to participate in class discussions and to express their own ideas, as well as to encourage 
them to get further assistance from teachers. 

Dimension 5: Individual Rapport (4 items): It evaluates the ability of a teacher to provide some spaces to the 
students by taking into account individual differences among students and his or her readiness to provide help 
and supports to the students. 

Dimension 6: Breadth Of Coverage (4 items): It measures whether the teacher discusses the various views and 
whether he or she is comparing the various theories in his or her teaching. 

Dimension 7: Assessment (3 items): This scale assesses the quality, equality, values and appropriateness of the 
teacher’s feedbacks and tests on the subject’s objectives. 

Dimension 8: Assignment (2 items): This scale asks the students to give values to the texts and additional 
readings provided by the teacher, and to evaluate the contribution of the tasks in the appreciation and 
understanding of the subject. 

Dimension 9: Overall Rating (2 items): This is the overall rating of the items. One of the items asks about the 
teacher’s rating in overall, while the other item asks about the subject’s overall rating. 
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The original version of SEEQ contains the scale of the 10th category other than the earlier 9 dimensions which is 
the criteria of the subjects/students. However, it is an additional scale (subscale) of the subject’s or the student’s 
background factor and it is not considered to be included in the evaluation of education (Al-Hinai, 2011). 
Therefore, this analysis does not include the 1 Oth category as it is out of the context of this study. Moreover, the 
scale was formed specifically for the subjects in higher education, not for the secondary level which is the focus 
of this study. 

4. Research Methodology 

Problem statement has mentioned the selection of SEEQ as the instrument for evaluation and control of the 
Arabic Language education in Malaysia. Thus, the purpose of this research is to evaluate the use of SEEQ in the 
context of Malaysian education. This research covers the validity and reliability of SEEQ based upon the 
collected local data. Hence, the Confirmatory Factor Analysis (CFA) is implemented. Harrington (2009) 
suggested that’ ...CFA can be used to examine whether the original structure of the measure works well in the 
new population In this research, CFA is used with the assistance of Analysis Of Moment Structure (AMOS) 
version 18. To test the internal consistency, Cronbach’s alpha is used for various scaled instruments such as 
SEEQ by using SPSS software. The evaluation of the usability of SEEQ in CFA is implemented based on fit 
indices such as CMIN/df, CFI, GFI, AGFI and RMSEA readings. This study used the following analytical 
readings as proposed by Hair, Black, Babin, and Anderson (2010): 


Table 1. CFA fit indices readings (Hair et ah, 2010) 


No 

Fit Indices 

Proposed Analysis 

1 

CMIN/df 

<3 good, < 5 sometimes permissible 

2 

CFI 

> 0.95 great, > 0.90 traditional, > 0.80 sometimes permissible 

3 

GFI 

>0.95 

4 

AGFI 

>0.80 

5 

RMSEA 

< 0.05 good, 0.05 - 0.10 moderate, > 0.10 bad 


This study also use the following guidelines by L. Cohen, Manion, & Morrison (2013) for the analysis of internal 
consistency index. 


Table 2. Cronbach’s Alpha internal consistency index readings 


Cronbach’s Alpha value 

Indicator 

>0.90 

Very high consistency 

0.80-0.90 

High consistency 

0.70-0.80 

Moderate consistency 

0.60-0.69 

Minimum consistency 

<0.60 

Low consistency 


A total of 511 respondents from national religious secondary school (SMKA) all around Malaysia have 
participated in this research. The respondents in this study were chosen randomly from 13 schools that were 
selected based on cluster sampling. Cluster Sampling was an ideal sampling method in order to ensure that the 
samples fully represented the whole research population across Malaysia. Zones or states were used as the basis 
for determining the study’s location sampling. However, only 491 respondents retained after data cleaning is 
done. According to Kline(2005), a sample that exceed 200 may be acceptable to most of the measurement model. 

5. Findings 

The first hierarchy in the Confirmatory Factor Analysis (CFA) is applied to the SEEQ model using AMOS. A 
total of 31 items which represent 8 constructs was drawn on AMOS canvas with the expected correlation 
between the 8 constructs. Fit indices of the first CFA shows that; CMIN/df = 2.589, CFI = 0.916, GFI = 0.874, 
AGFI = 0.846 and RMSEA = 0.057. Diagram 1 shows the SEEQ model at the primary stage. According to the 
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suggested reading analysis by Hair et al. (2010), the initial model does not show a perfect fit with Malaysian data. 
Therefore, 6 modification steps were performed to acquire appropriate index reading. These modification steps 
are based on the recommendation of Hair et al. (2010)namely, (1) examine the Factor Loading of each item, (2) 
examine Standardised Residual, and (3) examine the modification index. 


Fitness Indexes 


1 . 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 


ChiSq = 1051.047 
df = 406 

ChiSq/df = 2.589 
GFI = .874 
AGFI = .846 
CFI = 916 
RMSEA = .057 
TLI = .904 
RMR = .027 
PC LOSE = .004 
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Diagram 1. CFA-SEEQ primary stage measurement model 


The 6 modification steps have removed 2 items which are question 1 and question 20 due to low Factor Loading. 
Question 1 = 0.37 and question 20 = 0.58. Since SEEQ is a well-known instrument, the level of the Factor 
Loading should exceed 0.60 (Zainudin Awang, 2011). The low level of the Factor Loading symbolizes the 
weakness of the representative items against a construct in the model. Throughout the modification steps, there 
are 4 other question items removed due to high correlation between the four pairs of items. These pairs of items 
are; 15 <=>el6, c60c7. e4<=>13 and c80c 11. These pairs of items have the highest correlation value among other 
items. The high value shows that there is a repetition or overlap ( redundant ) items in measuring the construct. 
Therefore, four items from each pair were removed based on the lowest Factor Loading. All steps were 
performed based on improvement in the fit indices reading. Refer to Tables 3 and 4. 
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Table 3. Summary of the modification steps and the fit indices improvement 


Test 

Chi-Sq 

Df 

CMIN/df 

CFI 

GFI 

AGFI 

RMSEA 

Item Removed 

Model 

1051.047 

406 

2.589 

0.916 

0.874 

0.846 

0.057 

- 

Model 1 

1005.014 

377 

2.666 

0.918 

0.876 

0.847 

0.058 

Question 1 

Model 2 

924.469 

349 

2.649 

0.922 

0.881 

0.851 

0.058 

Question 20 

Model 3 

828.915 

322 

2.574 

0.929 

0.890 

0.861 

0.057 

Question 13 

Model 4 

738.004 

296 

2.493 

0.935 

0.900 

0.872 

0.055 

Question 5 

Model 5 

651.374 

271 

2.404 

0.941 

0.907 

0.880 

0.054 

Question 11 

Model 6 

540.991 

247 

2.190 

0.952 

0.919 

0.893 

0.049 

Question 7 


Table 4. Summary of the items removed throughout the modification 


Code 

Error 

Construct 

Question Statement 

Removed 

in 

Question 

1 

El 

Learning Value 

(EDU) 

1 have found the course intellectually challenging and 
stimulating 

Model 1 

Question 

E22 

Individual Rapport 

Teacher was adequately accessible to students during 

Model 2 

20 

(PI) 

office hours or after class 

Question 

13 

E15 

Group Interaction 
(IK) 

Students were encouraged to participate in class 
discussions 

Model 3 

Question 

5 

E6 

Teacher 

Enthusiasm (LE) 

Teacher was enthusiastic about teaching the course 

Model 4 

Question 

E18 

Organisation 

Proposed objectives agreed with those actually taught 

Model 5 

11 

(ORG) 

so I knew where the course was going 

Question 

E8 

Teacher 

Teacher enhanced presentations with the use of 

Model 6 

7 

Enthusiasm (LE) 

humour 
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Fitness Indexes 


1. 

2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 


ChiSq = 540.991 
df = 247 

ChiSq/df = 2.190 
GFI = .919 
AGFI = .893 
CFI = .952 
RMSEA = .049 
TLI = .942 
RMR = .022 
PCLOSE = .574 
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Diagram 2. SEEQ final model 


The internal consistency reliability test on the final model is done using the Cronbach s alpha analysis. 
According to Idris (2010), a complex construct uses various measuring items to measure the same construct. 
Hence, the internal reliability of all the items were measured as all items have the same role. After the final 
model is acquired, the data is added into SPSS for Cronbach’s alpha analysis. As for the internal consistency test, 
Cronbach's alpha is used for various scaled instruments such as SEEQ and Semantic Differential or 
Kuder-Richardson for instrument with dichotomy item. Researchers have done separate analysis for each 
construct and an overall analysis as done by al-Hinai(2011). The result of the analysis is as shown in Table 6 
Findings of Cronbach s alpha's Internal Consistency. 
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Table 5. Findings of Cronbach’s Alpha’s internal consistency 


No 

Code 

Category 

N 

Cronbach’s Alpha Value 

1 

A1 (EDU) 

Education 

4 

0.802 

2 

A2 (LE) 

Excitement 

3 

0.735 

3 

A3 (ORG) 

Organisation 

3 

0.817 

4 

A4 (IK) 

Group Interaction 

3 

0.792 

5 

A5 (PI) 

Individual Relationship 3 

0.825 

6 

A6 (CVR) 

Coverage 

4 

0.828 

7 

A7 (EXM) 

Examination 

3 

0.820 

8 

A8 (WRK) 

Assignment 

2 

0.784 

9 

A1 -A8 

Overall Scale 

25 

0.939 


Overall Cronbach s alpha analysis shows that the instrument is at a high internal consistency which is 0.939. The 
value obtained is almost as similar as the one obtained by Marsh (1982), Marsh and Roche (1993) and Al-FIinai 
(2011). On the other hand, Cronbach’s alpha analysis which is done separately for each construct shows quite a 
different value than the overall value. These differences are the result of the Cronbach s alpha index that is 
influenced by number of items in a single scale. Therefore, Pallant (2011) suggested that the 0.7 alpha index 
value or higher is better for a scaled instrument that has ten or more items. On the other hand, a 0.5 alpha value 
is good for a less than ten items scaled instalment. The alpha value of each construct in this instalment is 
considered as very good since they are higher than 0.735 with only 2 to 4 items. 

6. Conclusion 

The final SEEQ model retains 8 original construct i.e.; learning value, teacher enthusiasm, organisation, group 
interaction, individual rapport, breadth of coverage, assessment and assignments. A total of 25 items were found 
to be measuring well the 8 constructs with the Factor Loading exceeds 0.60. The analysis of the fit indices shows 
a good construct validity and stability. In addition, the reliability of SEEQ from the internal consistency aspect is 
high from 0.735 to 0.828 and the overall alpha value is 0.939. The overall findings support the use of the final 
SEEQ model for future evaluation of the quality of the Arabic Language education in Malaysia that uses the 
same or almost similar samples. It needs to be reminded that the final model is resulted from the exploratory 
nature of modification rather than confirmation. Therefore, it is suggested that the final SEEQ model is to be 
tested again for consolidation (Flarrington, 2009). Researchers expect with the continuation of this research or 
suchlike will help to improve the quality of the Arabic Language education in Malaysia. 
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